Gestural Auditory and Visual Interactive Platform
نویسندگان
چکیده
This paper introduces GAVIP, an interactive and immersive platform allowing for audio-visual virtual objects to be controlled in real-time by physical gestures and with a high degree of intermodal coherency. The focus is particularly put on two scenarios exploring the interaction between a user and the audio, visual, and spatial synthesis of a virtual world. This platform can be seen as an extended virtual musical instrument that allows an interaction with three modalities: the audio, visual and spatial modality. Intermodal coherency is thus of particular importance in this context. Possibilities and limitations offered by the two developed scenarios are discussed and future work presented.
منابع مشابه
Auditory Gesture for Interactive Installation Environments
Auditory gestures are sound signals that represent spatially located information that can be stored or accessed by human gestural interaction in space. This document presents initial work to develop a model of auditory gesture for application in interactive installation environments, also applicable to spatial interaction with responsive spaces in buildings and public works. An interactive inst...
متن کاملA.i. in Real-time (automated and Interactive) Sonification (of Gesture Data) Generative Real-time Sonification A.i. in Real-time (automated and Interactive) Sonification (of Gesture Data)
Drawing on implementations of A.I. models for real-time music performance that produce an immediate sonification and visualisation display from gestural and audio input, this paper concerns three areas of relevance to a broader array of inter-domain implementation: integration of A.I. modules (Artifical Life, biologically-inspired and Evolutionary processes) in a system for real-time computatio...
متن کاملTime Evaluation for the Integration of a Gestural Interactive Application with a Distributed Mulsemedia Platform
متن کامل
Gesture-Controlled Interaction with Aesthetic Information Sonification
Information representation in augmented and virtual reality systems, and social physical (building) spaces can enhance the efficacy of interacting with and assimilating abstract, non-visual data. Sonification is the process of automatically generated real time information representation. There is a gap in our implementation and knowledge of auditory display systems used to enhance interaction i...
متن کاملStudies of the mcgurk effect: implications for theories of speech perception
Studies of the McGurk effect demonstrate that observers integrate auditory information with visual information from a talker’s face during speech perception. The findings from these studies pose challenges for theories of speech perception that must account for how and why the auditory and visual information are integrated. One theoretical issue concerns the objects of speech perception. Some r...
متن کامل